This is Hacker Public Radio episode 4,084 for Thursday the 28th of March 2024.
Today's show is entitled, Cloud Learning.
It is hosted by Daniel Person and is about 10 minutes long.
It carries a clean flag.
The summary is, my experience trying to train a model online.
Hello Hacker's and welcome to another episode, Daniel here and today I'm going to talk
about Cloud Learning.
I'm using a machine learning model and training it in the cloud.
This was a topic that I went into this Christmas break because I was fed up with the
advent of code.
I couldn't really bear doing more of that, so I needed it at a topic to look into.
So I said, okay, I have this model, whatever it is, in this case it was a TTS model.
I wanted to train a voice to speak a particular language and create something that I could
use later on.
So I wanted to need to find somewhere in the cloud to train this model.
I have figured out a training the model on my own, on my computer, will take about eight
days to run through a full training cycle.
But I looked online, I could find places where I could train it in eight hours or in
10 hours, 12 hours and so on, depending on which graphics cards I were using, depending
on how many of them I was running and so on.
So I wanted to try to use this on in the cloud.
And I was talking to different cloud vendors.
So I reached out to Microsoft, looked at their solution.
Sadly, after two hours of video watching, trying to learn their platform and how to set
things up and where to go and what tools to use, I gave up because I still think Microsoft's
way of structuring things is not intuitive to me.
It's very confusing, so I couldn't really get into that.
I didn't want to spend more time on it.
Two hours trying to start even figuring out what tools to use is not fun for me.
So I also tried Google.
It was really easy to find my way there and I figured out that I wanted to use Vertex AI.
What I wanted to use is the model that I already had.
So I wanted to train that.
A lot of days kind of cloud providers gives you these kind of notebooks that you should
put in your model and run it there.
But this model was so complex that I needed to check out to get repository or run a
Docker image.
But in Vertex AI, you could run your own Docker images and connect them to cloud storage,
which was not that complicated actually.
You could do that pretty simply and I set up something that could train and then I
wanted to run it on some GPU power and there were the problem because Google don't give
you an graphics cards if you don't ask them for it.
So you needed to sign up and ask for a graphics card.
So on Christmas day, I asked for a graphics card.
I asked for one graphics card of one type and four graphics card of an older type.
And it took me about four weeks, three to four weeks until they actually gave me access to one
of these cards.
I haven't been able to run any jobs on that card yet.
I still am still trying to figure that one out.
But just asking for a card and that taking that much time was not really a super good experience.
They said that it should take about two to three business days, not four weeks.
But still I got access.
Then I went over and looked at Amazon because I want to do of course try all of the big ones.
And Amazon, frankly, just said no, you will not get any GPU power at us in our tooling.
You need to use SageMaker.
So SageMaker is pretty much use a notebook and train on GPUs in SageMaker.
But you still need to ask for GPUs so I could be declined there as well.
But you can't run your custom images.
So what I wanted to do, I was not allowed to do.
They just said no, which was really a really sad.
So I couldn't really use their service as well.
Then I found this service that was pay as you go and they were pretty much GPU as a service.
And I'm going to release a video about them.
I'm not sure if I actually can talk about the company.
But if you're following my channel, you will see that video eventually when we have released that.
So I'm talking and working with them around this video.
It's a very early company, a early concept.
So they haven't really released everything yet.
But I think that their way of doing it is really interesting and the right way to do it.
They have two different concepts pretty much or three different concepts.
They have an environment where you can say I'm want to do things in this data center pretty much.
So you have an environment and you can say that I want to run it this in Norway for instance.
Then when you have set up that environment, you could create these kind of volumes where you put your data.
So I created a volume of 100 gigabytes where I put all my data and my operating system and so on.
And then you could start virtual machines.
So I started a virtual machine with just CPU power and my volume.
I installed all the different dependencies and so on that I needed with the CPU power, which was very
cheap. So it was a cheap way of getting my dependencies and my model and all my data and so on
into my model set up and ready to do some training.
Then I shut down that CPU powered machine and I took a machine where they had
different amount of graphics cards. So they were running a
4,5,000, 6,000, a 100s, a 1,100s, a
H100s and L40s. So I had a bunch of these kind of cards and I have machines that had one card,
two cards, four cards and eight cards. So some of the machines were super powerful in order to train a lot.
But of course it become more expensive to run with a lot of cards, but it was still affordable.
I would say. So if you're running a load and you really want it to be done quick,
you just put more GPUs on it and then start it up with your volume that you have prepared.
Run your data load and then shut it down. Perhaps start a CPU load again in order to download the
the thing. And the best thing of this is that you either run it directly in your Linux environment
in this virtual machine or you could go into a VNC house and just look at it and run your things there
or give it its own IP on the internet and login using SSH so you had full access to the machine.
You could do whatever you want on these machines. So and just used GPUs. So it was GPU as a service
in its purest ways. So I really liked that approach to train things online.
I haven't found any other service that does it similarly and as well. There is of course the
option of running in Linux which I can buy as bought up now. They have similar solutions but they're
very expensive. What I've found so far. You also could run it in a digital ocean. I looked at those.
They are very interesting. When you start there you need first give them five bucks in order to just
sign up and when you had to sign up then you could get access to actually run things but you had to
ask for machine power again and the only accepted to run notebooks. I had to pay them five bucks
and I couldn't use them. When it actually came down to it. So yeah I can never see those five
bucks again because I can't use them. So they said okay pay us five bucks and you will use that for
your training later on but because I will never train with them I just gave them five bucks pretty
much. So this is what I have experienced when I have tried to train machine learning tasks
or models online using GPUs. Have you tried to do this and did you have a different experience?
Perhaps you have tried Microsoft and found it very easy and could give me some hints. Perhaps
record an episode explaining how to do that so I can figure it out myself as well. Or if you have
any other experience please share with the rest of the community. I'm very interested in this topic.
I hope that you liked this episode and I hope to see you in the next episode.
You have been listening to Hecker Public Radio at Hecker Public Radio.org.
Today's show was contributed by a HBR this night like yourself. If you ever thought of recording
podcast, click on our contributally to find out how easy it means. Posting price we are
has been kindly provided by an onsthost.com, the internet archive and our synced.net.
On the satellite stages, today's show is released on our creative comments,
attribution for going to international license.